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PREFACE 

AWS  forecasting  units  must  have  programs  that  routinely 
evaluate  the  quality  of  forecasts  and  local  point  warnings. 
Technical  improvement  programs  are  also  needed  to  maintain 
and  to  improve  the  technical  quality  of  forecast  products. 
There  are  many  factors  which  can  contribute  to  a decline  in 
forecast  performance.  Unit  managers  need  to  be  able  to 
define  the  causes  of  deteriorating  performance  and  to  imple- 
ment necessary  corrective  actions.  This  technical  note 
describes  a methodology  that  unit  managers  can  employ  to 
define  forecast  problems,  likely  causes,  and  actions  required 
to  correct  problems. 
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I.  INTRODUCTION 


In  1976,  1WW/DON  prepared  a worksheet  to  use  when  prepar- 
ing for  Special  Technical  Consultant  Visits  (TCVs)  to  units 
with  significant  forecasting  deficiencies.  Prior  to  a visit 
to  a unit,  that  unit  was  forwarded  a Potential  Problem  Analy- 
sis Worksheet  (see  Figure  1).  The  unit  was  asked  to  complete 
and  return  the  worksheet.  The  purpose  of  this  was  to  help 
the  unit  and  1WW/D0N  better  understand  the  nature  and  causes 
of  the  forecast  problem.  This  action  helped  1WW/D0N  decide 
how  to  prepare  for  the  Special  TCV  and  what  assistance  should 
be  provided  during  the  visit.  The  remainder  of  this  technical 
note  will  describe  a problem  analysis  methodology  recommended 
for  use  by  bas^‘‘s!£ktion  managers  whenever  an  adverse  trend 
in  forecast  or  local  point  warning  performance  is  observed. 


II.  DESCRIPTION  OF  THE  METHODOLOGY  FOR  ANALYZING  FORECAST 
PROBLEMS 

Deterioration  in  forecast  or  local  point  warning  perform- 
ance is  normally  detected  through  the  local  evaluation  program 
or  review  of  TAFVER  summaries.  Comparison  of  unit  performance 
versus  locally  developed  or  higher  headquarters  prepared 
performance  standards  clearly  reveals  substandard  performance 
and  adverse  trends.  Corrective  actions  are  often  implemented 
with  little  thought  given  to  clearly  defining  the  problem, 
to  listing  possible  causes,  and  to  determining  the  most  effi- 
cient way  to  correct  the  problem.  Often  we  react  based  on 
intuition  or  gut  feeling  rather  than  intelligent,  thoughtful 
evaluation  of  pertinent  facts  and  factors.  The  reaction 
response  frequently  results  in  an  immediate,  temporary  tech- 
nical fix  followed  later  by  problem  recurrence  and  another 
series  of  trial  and  error  attempts  to  cure  the  problem.  These 
type  efforts  are  costly  considering  the  man-hours  misused. 
Fruitless  and  prolonged  discussions  at  forecaster  seminars 
frequently  take  place.  These  can  leave  participating  fore- 
casters aggravated  and  wondering  why  so  much  of  their  time 
was  blown.  Wasteful  gathering  of  masses  of  data  for  forecast 
studies,  etc.,  sometimes  takes  place  in  this  reaction  type  of 
response  to  a deterioration  in  forecast  performance.  The 
impulse  to  attack  a forecast  problem,  without  first  systemati- 
cally evaluating  the  adverse  performance  trend,  must  be 
avoided . 

The  problem  analysis  methodology  shown  in  Figure  2 is  a 
way  to  replace  intuitive  problem  solving  with  a more  rigorous, 
systematic  procedure. 

The  three  bits  of  information  at  the  top  of  Figure  2 are 
self  explanatory. 


POTENTIAL  PROBLEM  ANALYSIS  WORKSHEET 


Unit /Location: 


Date  of  Analysis: 


PROBLEM  DEFINITION: 


:imes 


3,  6,  12,  24,  All  Hours 

Seasonal 

Cig/Vsby 

Optimism/Pessimism 

TRW,  RASH,  FG,  K,  Haze,  Etc._ 

TAF  vs  Persistence 

TAF  vs  CC 

Trends 

Which  Categories  

What  Verification  Times  CLoca' 

OEP 

PEP 

LAP 

OESE  

Manning  Shortage 
Leaves 

TDY  

Not  Enough  Assigned 
New  Personnel 

Training 

Experience  

Time-on-Station 

All  Forecasters 

Radar  Available/Used 
Climo  Data  Available /Used 
Satellite  Data  Available /Used_ 
Rawinsonde  Data  Available /Usee 

Too  Many  Visitors 

Abnormal  Weather 

Implemented  New  Program 

Workload  Changes 

TAF  Worksheet  

Incorporation  of  All  Data 
Tech  Studies  Available /Used 

ROT  Available/Used 

Forecaster  Seminar 

Self  Inspection 


*/* **  2.  PROBLEM  STATEMENT: 


**  3.  POSSIBLE  CAUSES: 


**  4.  POSSIBLE  REMEDIAL  ACTION (s) 


**  5.  TECHNICAL  CONSULTANT  RECOMMENDATIONS  OR  HIGHER  HEADQUARTERS 
ASSISTANCE  REQUIRED: 

* Based  on  entries  in  Item  1 , describe  what  the  problem  is 
and  when  it  began  and  summarize  contributing  factors. 

**  If  insufficient  space  is  provided  for  an  entry,  use  an 
attachment . 

Figure  1.  Example  of  Potential  Problem  Analysis  Worksheet 


PROBLEM  ANALYSIS  WORKSHEET 

Unit /Location: Date  of  Analysis: Analyst: 

REASON  FOR  ANALYSIS:  (Explain  in  terms  of  when  problem  began  and  trend  of  deviation  from 

performance  standard)  
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PROBLEM  STATEMENT: 


In  documenting  the  "Reason  for  Analysis,"  the  problem 
analysis  will  get  off  to  a proper  start  if  specifics  can  be 
determined  and  recorded.  The  specifics  should  include  such 
things  as  the  date  or  time  the  problem  began  and  verifica- 
tion statistics  that  objectively  show  the  current  level  and 
trend  of  the  substandard  performance. 

In  the  "Problem  Definition"  section,  you  could  line 
through  those  items  that  do  not  help  define  the  problem.  We 
suggest  use  of  a pencil  because  later  in  the  analysis  you 
may  determine  that  problem  definition  is  incomplete.  Blanks 
were  purposesly  left  below  each  column  so  you  can  add  any 
items  needed  to  help  define  the  problem.  In  the  far  right 
column  entitled  "Summary"  we  recommend  you  write  in  all  the 
items  not  lined  through.  The  problem  definition  step  will 
help  you  identify  what  the  problem  is  and  is  not. 

After  completing  the  "Problem  Definition,"  you  should 
be  ready  to  write  the  "Problem  Statement."  The  problem 
statement  should  be  a concisely  worded  statement  summarizing 
the  things  that  were  identified  in  the  problem  definition 
step . 

In  the  "Impact  to  Customer"  section  you  should  state 
how  the  defined  problem  is  affecting  customer  missions.  In 
Section  B of  your  terminal  forecast  reference  notebook  (TFRN) 
you  may  have  information  that  will  give  you  thoughts  on  what 
to  enter  in  this  section  of  the  problem  analysis  worksheet. 

In  Section  B of  your  TFRN  all  customer  activities  signifi- 
cantly affected  by  weather  should  be  discussed  along  with 
customer  actions  that  occur  whenever  significant  weather 
criteria  are  actually  exceeded  or  forecast  to  be  exceeded. 
Also,  the  costs  to  the  supported  organization  when  adverse 
weather  occurs  should  also  be  specified.  The  reason  for  this 
step  in  the  problem  analysis  is  to  make  you  assess  the  impact 
to  the  customer  associated  with  this  particular  forecast 
problem . 

With  the  resource  limitations  that  exist  in  base  weather 
stations  and  elsewhere,  we  must  devote  available  time  and 
expertise  to  the  most  important  forecast  problems.  For  this 
reason,  we  should  assign  priorities  to  forecast  problems  in 
a way  similar  to  the  duty  priority  lists  that  are  posted 
behind  the  briefing  counter.  We  simply  do  not  have  the  time 
and  resources  to  attack  all  problems.  We  must  be  selective 
and  choose  to  work  on  those  that  impact  customers  the  most 
(i.e.,  are  the  most  costly  in  terms  of  mission  degradation 
and/or  misuse  of  available  flying  hours,  etc.  These  thoughts 
dovetail  with  the  policy  statement  in  AWSR  80-3  which  indi- 
cates that  technical  improvement  efforts  of  units  with  a 
local  forecasting  function  must  be  directed  toward  forecast 
requirements  which  impact  supported  customers  operations. 


In  the  next  part  of  the  analysis  we  encourage  your  evalua- 
tion of  the  use  of  a number  of  things  that  are  important  in  the 
forecast  preparation  routine.  Evaluation  of  the  use  or  value 
of  each  thing  listed  should  be  in  terms  of  what  the  problem  is. 

In  other  words,  for  each  entry  in  the  "Problem  Definition" 
section  in  the  column  headed  "Summary,"  consider  whether  and 
how  centralized  products  or  the  manner  in  which  they  are  locally 
processed  and  used  may  have  caused  the  problem.  Then  describe 
ways  to  improve  the  use  or  value  of  centralized  products.  By 
doing  this,  you  have  begun  to  list  things  that  should  be  done 
to  correct  the  problem.  After  completing  that  action,  then 
consider  whether  and  how  the  local  analysis  program  or  any  local 
analysis  procedure  deficiencies  contributed  to  the  entries  in 
the  "Problem  Definition"  section.  Repeat  the  other  actions 
described  above.  Again  space  is  left  under  the  section  entitled 
"Evaluate  Use  or  Value  Of"  for  you  to  add  important  items  we 
may  have  overlooked.  This  will  be  one  of  the  more  difficult 
sections  of  the  problem  analysis  to  complete.  Meticulous 
attention  to  this  section  will  enable  you  to  uncover  the  most 
likely  causes  of  the  forecast  problem. 

In  the  "Problem  Causes"  section,  the  reason  we  call  for 
documentation  of  the  "possible"  and  "most  probable"  causes  of 
the  problem  is  because  we  don't  want  you  to  discard  a "possible 
cause"  that  after  additional  thought  and  analysis,  could  turn 
out  to  be  one  of  the  "most  probable"  causes. 

After  stating  or  listing  the  "Problem  Causes,"  describe 
those  actions  necessary  to  correct  the  current  situation  and 
reduce  the  possibility  it  may  recur.  You  should  have  determined 
all  the  required  corrective  actions  in  the  "Evaluate  Use  or 
Value  Of"  section.  If  possible,  list  the  required  actions  in 
order  of  importance.  It's  appropriate  at  this  time  to  think 
about  who  will  be  responsible  for  implementing  each  action  and 
when.  Progress  reports  should  be  prepared  on  a periodic  basis 
to  determine  the  timeliness  and  appropriateness  of  corrective 
actions.  If  progress  is  unsatisfactory,  the  previously  completed 
problem  analysis  worksheet  should  be  reviewed  to  see  if  any  of 
the  analysis  was  erroneous.  Progress  reports  could  be  prepared 
in  memorandum  format  and  attached  to  the  problem  analysis  work- 
sheet. The  overall  "Results"  of  the  corrective  actions  taken 
should  be  recorded  when  all  actions  have  been  taken.  The 
analysis  and  progress  report  should  be  retained  in  case  the 
problem  raises  its  ugly  head  again  sometime  in  the  future. 

We  won't  be  offended  if  you  don't  implement  this  problem 
analysis  methodology  when  a decline  in  forecast  performance 
occurs  at  your  unit.  Our  main  purpose  for  this  technical  note 
is  to  encourage  you  to  organize  your  thought  process  when  you 
accomplish  an  analysis  of  a forecast  problem.  The  next  time 
you  are  confronted  with  a forecast  problem,  pull  this  technical 
note  from  the  dusty  corner  of  your  terminal  forecast  reference 
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file  bookcase.  If  you  choose  not  to  apply  the  problem  analysis 
methodology  we  have  described,  at  least  structure  your  thinking 
so  you  can  logically  derive  prudent  decisions.  If  this  techni- 
cal note  helped  you  in  any  way  to  do  this,  the  effort  to  prepare 
this  document  was  well  worth  it. 


